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INTRODUCTION TO THE PROCEEDINGS 



Interest in new instructional methods has been particularly strong and 
widespread throughout the region served by the Far West Laboratory for 
Educational Research and Development. The professional climate and the 
relatively high financial support for education within this region have 
attracted many educational innovators to the public schools and have placed 
many schools in the region at the forefront in field exploration and innovation. 

The need for an instructional methods conference was based on the premise 
that interaction among educational leaders in the region would lead to an even 
higher rate of exploration and innovation as these leaders learned from each 
other. It was felt that the impact of such a conference could be even greate* 
if leaders in the region had an opportunity to interact with a panel of 
researchers who were involved in interesting and promising explorations at 
different points along the current frontier of knowledge of instructional 
methods. Thus, two major criteria were employed in identifying prospective 
panel members. First, the person should be actively engaged in a promising 
research program related to the broad area of instructional methods. Second, 
the work of each panel member should be in a markedly different area from 
that of other panel members in order to provide the widest possible range of 
ideas and viewpoints. I believe that you will agree, upon reading the 
following summaries of the conference presentations that the Laboratory was 
extremely successful in assembling a panel that met these criteria. 

Although all of the techniques discussed are still at an exploratory 
s age of development, they have produced results ranging from promising to 
some that can only be described as spectacular. Those of us at the Laboratory 
have concluded from the enthusiastic response of the school leaders who were 
present that the conference was successful. Although we realize that the 
following summary of the conference proceedings is a poor substitute for the 
process of interaction that was possible at the conference itself, we believe 
that even this rather brief report has something of value for any thoughtful 
and forward looking educator. 



Walter R. Borg, Director 
Instructional Methods Program 
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COMPUTER ASSISTED INSTRUCTION: A TOOL FOR TEACHING AND RESEARCH 

H. A. Wilson 



Senior Research Associate 
Stanford-Brentwood CAI Project 



Introduction 

The title of this essay is loisleading. What I am actually going to 
discuss is team teaching. ' The team I will describe is large and one of 
their tools is new and very powerful. The tool I refer to is a general 
purpose information processor and its peripheral equipment, more commonly, 
known as a computer assisted instructional system. 

The tool is so new and glamorous it tends to draw attention away from 
the people who are using it and the purpose for which it was developed. I 
want, therefore, to turn your attention to the people on the teaching team 
and to a brief statement of their goals and purposes before proceeding to a 
description of the system itself. 

The Teaching Team 

Two. /Outstanding educator-scholars furnish joint direction for the team. 
Professor Patrick Suppes is one of the nation *s leading philosopher-mathe- 
maticians and Professor Richard Atkinson is a. mathematical psychologist of 
international reputation. They lead a group of fifty linguists, mathematicians, 
educators, psychologists, artists, writers, programmers, school administrators, 
and classroom teachers. We are known as the Stanford CAI Project. 

The organization of teachers into Instructional teams is an Increasingly 
common* pattern in today's schools. Detailed planning of the lesson material 
and of the mode of presentation is a cooperative effort of the entire team. ' 
Individual members of the team have different responsibilities in the prepara- 
tion and presentation of the lessons. The evaluation of results, in terms o 
student performance and behavior, is made by the team as a whole, based on 
observations of each of the members. 

There are many similarities between current team teaching practices and 
the operations of the Stanford CAI Project. The differences that exist are 
largely differences of degree rather than of substance. We too have worked 
as a group in the detailed planning of the lesson material. Two years have 
been spent in planning and preparation. Every word, picture, and eudio 
message that the students will encounter on the system has been specified 
and prepared in advance. All conceivable responses that the student 
make to every question or problem have been accounted for. Individual members 
of the project team have had different responsibilities in the preparation 
and presentation of the lessons according to their unique talents and areas 
of expertise. Evaluation and revisions will be undertaken by the group based 
on individual observations and the detailed performance record of each student. 
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The Stanford CAI project is working toward two major goals: 1) to truly 

Individualize and optimize instruction in initial reading and mathematics, 
and 2) to investigate in a detailed fashion the processes through which young 
children acquire initial reading and mathematical skills and concepts. In 
order to progress toward these twin goals we have interacted extensively with 
industry in the development of a highly sophisticated instructional system. 

The CAI System 

The Stanford Project is not the first to investigate CAI applications 
to human learning. The PLATO system at tbq University of Illinois and the 
CLASS system at Systems Development Corporation are but two examples of 
pioneering work in the field. The Stanford Project is unique however, in 
that for the first time two major sections of the primary grades* curriculum 

are being taught for a full year under computer control in a public school 
setting. 

A complete IBM 1500 CAI system is housed on the grounds of the Brentwood 
Elementary School in the Ravenswood School District in East Palo Alto. The 
facility is called the Stanford-Brentwood CAI Laboratory. The laboratory is 
equipped with sixteen student terminals. One half of a normal classroom can 
be accomodated on the system at a time. There are four first-grade class- 
rooms at Brentwood. The students of two of the classrooms are involved in 
the reading programs and the students of the remaining two first-grade rooms 
are in the irtathematics program. Each student receives thirty minutes of 
instruction on the system every day. 

Each response terminal consists of a cathode-ray tube on which the 
computer can generate alpha-numeric characters and a limited set of graphics, 
a rear view projection device which is essentially a random access 16mm film 
strip projector under computer control, an audio unit which at the terminal 
end consists of a set of head phones and a microphone, a keyboard which can 
be used for student responses and for editing of text and programs, and a 
light pen which can be used to enter a response by touching it to the desired 
area on the face of the cathode-ray tube. 

The system which drives the response terminals consists of an IBM 1800 
central processing unit, six disc drives, two magnetic tape drives, a bank 
of sixteen random access audio tape units with both record and playback 

capabilities, a terminal control buffer, two proctor terminals, a card reader 
and a line printer. 

CAI: \ Tool for Teaching 

Computer assisted instruction has a theoretical basis, as indeed does 
all programmed learning, in the notion that immediate reinforcement facilitates 
learning. For the humcin learner, reinforcement comes through both verbal 
praise acting as a reward and simple knowledge of the results of one*s own 
actions. Correct responses yre rewarded in the reading program through such 
verbal messages as "good," "you’re doing fine," "right," etc. The standard 
reward in the math program consists of a visual reward of a smiling face 
appearing on the scope. Common sense and many learning studies all indicate 



that a constant stream of verbal or visual rewards for every correct response 
tend to lose their effectiveness. Accordingly, rewards in the reading program 
and in the math program are given on an intermittent basis. Immediate feed- 
back is provided through reward messages or the presentation of the next 
problem and also through wrong answer messages. 

Uninformed criticism of computer assisted instruction often carries a 
component of fear that the instructional process will somehow become de- 
humanized, that the students will in some sense become little automatons 
themselves. As it turns out, however, the elimination of the social inter- 
course aspect of learning through CAI is one of its great strengths. 

The computer is an eternally patient teacher. The machine never becomes 
angry or threatening. Those of us who have spent some years teaching in the 
classroom are well aware of the fact that after repeated errors by a student 
it is difficult, if not impossible, to restrain certain voice or facial cues 
which indicate our displeasure. The messages coming from the machine, however, 
are completely free of any such threat or anger. The wrong answer messages 
recorded in the quiet of the recording studio can be continuously a neutral 
"No, this is the right word. Touch it." 

The CAI system is also utterly impartial. It cannot be bribed by smiles 
and running errands. Neither does it differentiate amongst students by clean 

or messy desks. 

We have found that the concentration of even very young students can be 
maintained at a rather high level by appropriate pacing of the presentation 
of the materials and of the time-out limits. Their concentration is also 
increased, particularly in the tutorial system, by the use of partitions 
between the response terminals. The individual student is not distracted by 
the actions of his classmates, nor is he carried along by their responses. 

Each individual is truly independently responsible for interacting with the 

learning materials. 

The basic rationale, however, for the use of computer assisted instruc- 
tional systems, resides in their potential for individualizing instruction 
in a very real way. If there is any one fact which has been thoroughly estab- 
lished in sixty years of intensive educational investigation, it is that a 
wide range of individual differences will be found in any classroom or dimension 
one wishes to investigate. CAI offers us a tool for tailoring our instructional 
procedures to these Individual differences. 

The current paper and pencil programs can account for individual dif- 
ferences in learning time. A bright student who responds rapidly can execute 
many more frames and cover a greater ^ount of material in a given time interval 
than the slow student who responds quite deliberately. 

The nearly unlimited branching capability inherent in a CAI system allows 
the students to spread not only on the temporal dimension but also to take 
essentially different paths through the curriculum. Branching decisions for 
each sfudent in a CAI system may be contingent upon a single response, his 
past history of responses, response latency, or some combination of these 
considerations weighted by previously acquired psychometric data. 
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CAl! A Tool for Research 



The usefulness of CAI as a tool for research resides in two factors; 

1) the control of independent variables and 2) the detailed response data 
which is recorded by the machine. 

Educational research conducted outside the laboratory and in an existing 
school situation has long been plagued by the impossibility of controlling 
many variables inherent in the classroom and in the presentation of materials 
by the classroom teacher. CAI, in a sense, brings the laboratory into the 
school. The CAI laboratory at the Brentwood School has achieved a degree of 
control of environment and presentation that has been heretofore impossible 
in a classroom setting. The temperature of the terminal room is constant; 
the lighting in the terminal room is constant. The immediate environment 
of each student *s response terminal is precisely the same as any other student's. 
The chairs and the machines are identical for all students. Every picture seen 
in the projection device, every bit of orthography or other display on the 
scope, and every audio message whi^'h the student hears, have been previously 
specified and can be as standardized or varied as the experimenter desires. 

This is not to say, of course, that all sources of variation are controlled. 

The students are not placed in Ball jars between each session in the terminal 
room. The CAI facility does achieve, however, a degree of control equivalent 
to that of campus learning laboratories. Many problems in learning theory, 
which have been investigated rigorously only in a laboratory setting, can now 
be looked at in an on-going school context. 

The second capability of CAI whi h is extremely important for research 
is the collection of fine-grained response data. Allow me to give you an 
example of what I mean by fine-grained data; each response that every student 
makes to each problem he encounters in both the reading and mathematics 
curricula are permanently recorded on the data tapes. Each response record 
includes a complete description of the response by its coordinates on the 
face of the scope or the keys depressed on the typewriter. The response is 
defined as correct or incorrect, and if it is incorrect, it is categorized 
according to the type of 'error made. The response latency is recorded in 
tenths of a second. The contents of 30 registers and 29 switches associated 

with the student's past history of performance are recorded with each response 
on the data tape. 



An individual student makes a response every fifteen seconds on the average. 
During the period of an entire school year a single student will enter approxi- 
mately 216,000 responses. We are running one hundred students on the Brentwood 
system. Devising a sorting routine to handle this tremendous inflow of data 
is, in itself, no trivial task. 

Another important use of CAI as a research tool is found in the area of 
mathematical psychology, particularly in the area of mathematical learning 
models. The quantity and nature of the response data which may be gathered 
in the CAI system allows the mathematical psychologist to test his various 
models in a situation which is a much closer approximation of actual classroom 
•learning than existed in the past. Typically, such models have been tested 
through infra-organism behavior or through such contrived tasks as paired- 
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associate list learning or probability learning tasks* Professor Atkinson, 
who is co-director of the CAI project and principal investigator of the 
reading project, will use the kind of sequential response data which we will 
gather in the Brentwood facilities to develop optimization models for learning. 

CAI: A Tool for Curriculum Evaluation 

A final area of usefulness of CAI in the educational process is in cur- 
riculum evaluation. Our current methods of curriculum evaluation are extremely 
gross, relying on standardized tests or specially devised tests given on an 
intermittent schedule. The best that can be expected from such evaluation 
procedures is to be able to compare the general outcomes of one method or one 
curriculum approach to some other method or approach. Little or nothing can 
be said about the efficiency of any detailed section of the curriculum. It 
is exactly at this detailed level that CAI exhibits its greatest power for 
evaluation. The performance data gathered in the CAI system may be examined 
at all levels, from that of overall goals down through various strategies 
and approaches and then blocks of homogeneous problem types and,_^ further, 
through individual problem types to the individual problems themselves. At 
each level we can actually examine the students’ performance records to dis- 
cover if this section or level of the material is functioning in the manner 
for which it was designed. 

Implications 

The implications of CAI for education are truly revolutionary. A serious 
discussion of the probable future impact of CAI on current school organization 
and practices is far beyond the scope of this paper. However, one point of 
obvious importance must be briefly discussed. That point concerns the im- 
plications of CAI on the already changing role of the classroom teacher. 

It is clear that a large proportion of the teachers’ current respon- 
sibility for imparting facts, basic skills and concepts and providing routine 
drill can be more efficiently and effectively handled by the machine. This 
does not imply that the computer will replace the teacher. Rather, by 
utilizing the computer to attend to those tasks for which it is uniquely 
effective, the teacher will have the time and energy and information with 
which to plan and to carry out the programs of socialization and intellectual 
enrichment which are the unique domain of the human teacher. 

The extension of this view of the function of the classroom teacher in- 
to the field of teacher preparation will have far-reaching impact. Teachers 
must be trained to make the fullest use of data from a CAI system. They must 
also be given more than philosophical and moral precepts when they face the 
actual task of performing as humans and not as machines. 
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INSTRUCTION lOR EDUCATIONALLY AND CULTURALLY DEPRIVED ADOLESCENTS 

John T. Dailey 

Research Professor of Education 
The George Wash.’^ngton University 



Vocational and technical education today is facing two very serious 
problems. First, there is the growing competition with colleges for the 
upper abil.,ty high school graduates and even. average ability high school 
graduates. Because of the considerable overlap in academic and vocational 
ability patterns, as a higher and higher proportion of our youth enter 
college, the supply of youth with well-developed academic skills who want 
vocational training becomes less and less. At the same time that this is 
happening there is a growing need for vocationally and technically trained 
youth at high skill levels to meet the expanding demands of both industry 
and the military services for technically traineo personnels Vocational 
and technical schools and the Armed Forces find that they must educate 

many marginally trainable youth that is, young people who are train- 

able in some useful occupational skills but are not readily trainable for 
the more highly skilled jobs with a good future. These marginally train- 
able youth lack basic educational skills as well as basic understanding 
of mechanical and technical concepts and principles and other knowledge 
needed to take full advantage of opportunities for vocational and technical 
education either in school or on the job. Anything that can be done to 
increase the proportion of our young people who are capable of being trained 
for more highly skilled jobs represents an important contribution towards 
reducing the number of unemployed and underemployed among our culturally 
deprived adolescents and young people. 

If our culturally deprived adolescents are to have a full chance to 
emerge from poverty there ^must be a massive upgrading in the levels of their 
developed basic skills that we often call talents. Various studies of 
Project Talent* data have indicated that many of our youth from culturally 
deprived backgrounds leave schbol lacking the basic skills and aptitudes 
or talents necessary for training for many of our more highly skilled jobs. 

Of course, such tests as those in Project Talent do not identify latent 
talents that a student’s environment has not permitted to develop. They 
measure the skills that he has developed but not the skills he might have 
developed in a more favorable environment. Nevertheless, the important 
tests in Project Talent do define the skills that must be developed in our 
culturally deprived adolescents if they are to be able to compete on equal 
terms in our society with those from more fortunate .environments. 

In order to help every individual reach his full potential, new and 
better ways of assessing and developing the potentials of individuals from 
minority or culturally deprived groups are needed. New materials and methods 
are needed to identify and develop these latent talents. 



* Flanagan, J.C., et al.. The American High School Student . Technical 
Report to the U.S. Office of Education, Cooperative Research Project No. 
635. Pittsburgh: Project TALENT Office, University of Pittsburgh, 1964. 

' 8 










The Education Research Project of The George Washington University has 
attempted to meet this need by developing a curriculum and materials for 
teaching basic vocational talents to culturally deprived adolescents and 
young people. These materials are as follows: 

1) A series of paper-and-pencil basic aptitude practice exercises • 
designed to teach nonverbal abstract reasoning, basic mechanics, basic 
electricity, and both two- and three-dimension spatial reasoning. These 
materials are ve..y similar to test items in format and are designed for use 
in training both adolescents and young adults. 

2) A series of basic readers especially designed for students in grade 
8 who read at the 6th grade level of difficulty or lower. They provide read- 
ing materials at a low-difficulty and high-maturity level, and cover various 
aspects of tools, machines, devices, and occupations. They are to be used in 
conjunction with the basic aptitude practice exercises. The titles of the 
readers are: 

1. Transportation Long Ago 

2. Transportation Today and Tomorrow 

3. The Automobile 

4. Occupations for You 

5. Tools and Basic Machines 

3) Laboratory equipment and simple demonstration devices designed to 

teach 9th-grade students those aspects of mechanical ability and basic 
mechanical and technical comprehension and understanding that are not suit- 
able for teaching either by readers or paper-and-pencil exercises. This 
equipment is used in conjunction with the basic aptitude practice exercises. 
The laboratory equipment demonstrates such principles as: levers, gear 

trains, belts and pulleys, pendulums, inclined plane and screw threads, 
physical phenomena, and simple electrical circuits. Student and teacher 
manuals were also developed. 

As the development of the new curriculum and materials for teaching 
basic vocational talents in the 8th and 9th grades progressed, a number of 
additional ways of using the materials have been indicated, among which are: 

1) Use in adult basic education, literacy training or p re-vocational 
training. 

2) Use by the Armed Forces for training marginal personnel. 

3) Use in emerging nations to supply the basic technological skills 
necessary as a foundation for the training of engineers and technicians. 

4) Use in short concentrated courses to assess how well students learn 
yarious technological skills and concepts and how well they like learning 
them. On the basis of this, they can be guided into appropriate long-range 

training courses. 

5) Such concentrated programs are also appropriate for use in summer 
Talent Day Camps in Technology. These camps would be to assess students, 
sharpen their talents, and guide them into post-high school training programs 
in junior colleges and technical institutes. 





The primary purpose of the Talent Camps would be to give students a 
chance to learn the basic talents underlying success in training for techno- 
logical careers. On the basis of how well they learn the various basic 
technological skills and concepts and hyw well they enjoy learning them, they 
can be guided into existing high school, junior college, or other post-high 
school programs for technological careers. This could be a new approach to 
the problem of identifying and developing the latent talents of students from 
a wide variety of backgrounds, who may not have had an adequate chance to 
learn the skills and concepts sampled by existing occupational aptitude tests. 
In effect, the Talent Camp could serve as an aptitude test and also an Interest 
test. The students could discover their real talents and interests, and the 
school could assess their patterns of potential for success in advanced tech- 
nological training in vocational courses, junior college, or similar programs. 
The basic curriculum could be based on the materials developed by The George 
Washington University for teaching basic vocational talents. 

Talent Camps could stress the following types of activities: 

a. Swimming,. sports, music, arts and crafts 

b. Excursions to museums, hospitals, laboratories, factories, shops, 
junior colleges, and technological institutes 

c. Contact with visiting specialists with outstanding skills and 
experience in various areas of technology 

d. Reading stimulation and practice using The George Washington 
University occupational readers 

e. Practice with aptitude exercises designed for learning basic 
technological skills and concepts 

f. A laboratory course in basic mechanical, electrical, and 
electronics principles and phenomena 

g. Occupational and vocational counseling 

This procedure could enable schools to identify and strengthen the basic 
vocational talents of their students so that more of them can develop the 
basic skills essential for emergence from poverty. 

The materials developed were tried out and evaluated in a representative 
sample of 2,538 students in eight of the nation's high school systems (San 
Antonio, Texas; Atlanta, Georgia; Wise County, Virginia; Washington, D.C.; 

New York City; Bayonne, New Jersey; Erie, Pennsylvania; and Detroit, Michigan). 

Key Project Talent tests were used for pre-test and post-test. Results 
are shown in Tables 1 and 2. (See next page). 

On the post-testing at the end of the school year it was found that 
substantial post-test increases were found in subject-matter areas that were 
taught, but only normal growth or less was found in the areas not taught. 
Abstract Reasoning and Two- and Three- Dimensional Spatial Reasoning were 
found to be especially trainable. 

One very interesting finding was the apparent absence of any "general 
test-taking skill." Training in one skill does not affect test performance 
on a different skill. Little improvement was found in reading or arithmetic 
skills on the post- tests. 






In grade 8 less improvement was found in mechanical reasoning and infor- 
mation. Additional gains in the technological skill areas were found in the 
9th grade since the laboratory materials were used only at this level. 

One result of the training was to increase the estimated reliability of 
some of the Project Talent tests. For some schools, some of the tests had 
exceedingly low reliability. This reflected the extremely low performance 
level of the students in some of the groups. I'Jhenever a test is too difficult 
or too easy for a group, its reliability will be highly attenuated. Thus if 
a school group tends to do poorly on a test, raising its performance level 
should raise ‘the reliability of the test. 

The pre-tests and post-tests were separately subjected to factor analysis 
and these analyses were done separately by school systems. It was found that 
the factor structure of a test varies widely from school to school, and 
especially large differences in factor patterns were found for rural and 
urban groups. Apparently, what a test measures is heavily affected by the 
nature of the group and its environment. The same forces that cause over- 
and under- achievement on mechanical tests also seem to operate to change the 
factor content of tests. 

Factor analyses of the post-tests reveal a general tendency toward find- 
ing a more powerful "general" factor accounting for a higher proportion of 
variance than vjas found in the pre-test analyses. There were also some im- 
portant changes in the general factor structure. The groups that gained the 
most seem to have the greatest changes in the factor structure of the tests. 

The data indicate that aptitude test batteries do not seem to function 
normally with very low-performance-level culturally deprived adolescents. 

They do not inter-correlate the way they did when standardized and validated 
and thus cannot be properly interpreted by counselors and teachers. It is 
suggested that training in the subject-matter areas measured by the tests 
might result in making the tests function more nearly like they do for the 
general population and thus make possible more meaningful interpretation and 
use. This suggestion should be eiqjlored in future research.. 

It has been demonstrated that it is possible to teach several basic 
vocational talents directly. The experimental materials developed for this 
purpose have been revised and will be tried out in a number of additional 
training programs for adolescents and young adults. These materials are 
being released for general use by schools and training programs. 

It is planned also to carry out follow-up studies to relate the in- 
creases in performance on tests to increased trainability in vocational and 
technical training programs. 

Schools should give serious consideration to more emphasis on teaching 
basic skills, and research should be initiated to develop talent training 
materials for the full spectrum of important tests such as those in Project 
Talent. 



The work reported herein was carried out under Contract No. OE-5-85-023 
with the United States Office of Education, HEW. 




11 



Table 1 



Results of Tiyout of Talent Training Materials * 
(School Year 1965-66) 

8th Grade — I^oys 



Project 

Talent 

Ident. 

No. 


Name of Test 




No 


Mean- 


Mean2 


M 1 -M 2 

!!Il 


SDi 


SD2 


Pre- 

Test 


Post- 

Test 


Pre- 

Test 


Post- 

Test 


Pre- 

Test 


Post- 

Test 


R-290 


Abstract Reasoning 


A83 


4!)8 


6.352 


8.055 


.595 


2.86 


2.91 


R-270 


Mechanical Reasoning 


483 


458 


7.174 


8.900 


.506 


3.41 


3.72 


R-311 


Arithmetic Reasoning 


483 


458 


4.004 


4.790 


.342 


2.30 


2.58 


R-281 


Visualization — 2 Dimensions 


483 


469 


9.066 


14.309 


.935 


5.61 


6.12 


R-282 


Visualization — 3 Dimensions 


483 


469 


6.095 


7.367 


.482 


2,64 


3.13 


R-250 


Reading Comprehension 


483 


469 


15.251 


17.623 


.324 


7.33 


9.17 


R-102 


Vocabulary Information 


479 


449 


6.979 


7.898 


.246 


3.73 


3.89 


R-106 


Mathematics Information 


479 


449 


4.351 


5.200 


.371 


2.29 


2.97 


R-107 


Physical Sciences Info. 


479 


449 


5.447 


6.094 


.208 


3.11 


3.43 


R-108 


Biological Sciences Info. 


479 


449 


3.795 


4.178 


.172 


2.23 


2.28 


R-110 


Aeronautics & Space Info. 


479 


449 


2.871 


3.203 


.178 


1.86 


2.08 


R-in 


Elect *y & Electronics Infor. 


479 


449 


4.962 


5.786 


.245 


3.36 


3.46 


R-112 


Mechanics Information 


479 


449 


6.559 


7.461 


.265 


3,41 


3.49 





9th Grade 


— Boys 










R-290 


Abstract Reasoning 


384 


374 


7.375 


9.086 


' .547 


3.13 


3.12 


R-270 


Mechanical Reasoning 


384 


374 


8.844 


10.059 


.318 


3.82 


4.36 


R-311 


Arithmetic Reasoning 


384 


374 


5.362 


6.214 


.281 


3.03 


3,55 


R-281 


Visualization — 2 Dimensions 


387 


379 


10.395 


15.501 


.861 


5.93 


5.75 


R-282 


Visualization — 3 Dimensions 


387 


379 


6.972 


8.475 


.494 


3.04 


3.19 


R-250 


Reading Comprehension 


387 


379 


20.380 


22.815 


.248 


9.83 


10.75 


R-102 


Vocabulary Information 


384 


377 


8.698 


9.844 


.295 


3.89 


4.07 


R-106 


Mathematics Information 


384 


377 


5.773 


7.050 


.403 


3.17 


3.85 


R-107 


Physical Sciences Info. 


384 


377 


6.839 


8.284 


.382 


3.78 


3.80 


R-108 


Biological Sciences Info. 


384 


377 


4.740 


5.061 


.137 


2.34 


2.28 


R-110 


Aeronautics & Space Info. 


384 


377 


3.531 


4.180 


.286 


2.27 


2.31 


R-111 


Elect *y & Electronics Info. 


384 


377 


5.721 


7.329 


.481 


3.34 


3.97 


R-112 


Mechanics Information 


384 


377 


7.875 


9.393 


.441 


3.44 


3.80 



*NOTE: For school systems in Washington, D.C.; Atlanta, Georgia; Erie, Pennsylvania; 

San Antonio, Texas; and Wise County, Virginia. 
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Table 2 

Results of Tryout of Talent Training Materials * 
(School Year 1965-66) 

8th Grade — Girls 



Project 

Taletit 




Nl 


»2 


Meani 


Mean2 


M2-M1 


SBi 


SD2 


Ident . 




Pre- 


Post- 


Pre- 


Post- 


SD7" 


Pre- 


Post- 


No. 


Name of Test 


Test 


Test 


Test 


Test 


1 


Test 


Test 


R-290 


Abstract Reasoning 


120 


113 


4.517 


6.487 


.801 


2.46 


2.93 


R-270 


Mechanical Reasoning 


120 


113 


4.408 


5.407 


.409 


2.44 


2.54 


R-311 


Arithmetic Reasoning 


120 


113 


3.717 


4.195 


.187 


2.55 


2.82 


R-281 


Visualization — 2 Dimensions 


123 


120 


6.122 


10.825 


1.117 


4.21 


6.36 


R-282 


Visualization — 3 Dimensions 


123 


120 


4.740 


5.883 


.510 


2.24 


2.87 


R-250 


Reading Comprehension 


123 


120 


13.220 


15.558 


.360 


6.50 


7.85 


R-102 


Vocabulary Information 


122 


112 


5.049 


5.955 


;374 


2.42 


2.97 


R-106 


Mathematics Information 


122 


112 


4.066 


4.446 


.182 


2.08 


2.55 


R-107 


Physical Sciences Info. 


122 


112 


4.172 


4.670 


.234 


2.13 


3.01 


R-108 


Biological Sciences Info. 


122 


112 


3.041 


3.393 


.191 


1.84 


2.39 


R-110 


Aeronautics & Space Info. 


122 


112 


1.885 


2.089 


.156 


1.55 


1.58 


R-111 


Elect 'y & Electronics Info. 


122 


112 


3.344 


4.455 


.542 


2.05 


2.14 


R-112 


Mechanics Information 


122 


112 


4.246 


4.688 


.193 


2.29 


2.42 




9th 


Grade 


— Girls 










R-290 


Abstract Reasoning 


184 


182 


5.793 


7. .363 


.565 


2.78 


3.07 


R-270 


Mechanical Reasoning 


184 


182 


5.299 


6.121 


.345 


2.38 


3.03 


R-311 


Arithmetic Reasoning 


184 


182 


4.359 


4«758 


.175 


2.27 


2.74 


R-281 


Visualization — 2 Dimensions 


185 


181 


7.059 


12.475 


1.221 


4.43 


6.19 


R-282 


Visualization003 Dimensions 


185 


181 


5.492 


6.878 


.605 


2.29 


2.90 


R-250 


Reading Comprehension 


185 


181 


18.281 


19.862 


.195 


8.12 


9.22 


R-102 


Vocabulary Information 


181 


179 


6.840 


7.911 


.415 


2.58 


3.13 


R-106 


Mathematics Information 


181 


179 


4.624 


5.704 


.417 


2.59 


2.53 


R-107 


Physical Sciences Info. 


181 


179 


5.011 


6.324 


.503 


2.60 


2.75 


R-108 


Biological Sciences Info. 


181 


179 


3.856 


3.955 


.051 


1.96 


1.80 


R-110 


Aeronautics & Space Info. 


181 


179 


2.497 


2.782 


.198 


1.44 


1.55 


R-111 


Elect’y & Electronics Info. 


181 


179 


3.983 


4.693 


.325 


2.18 


2.28 


R-112 


Mechanics Information 


181 


179 


4.967 


5.905 


.392 


2.39 


2.60 



*NOTE: For school systems in Washington, D.C.; Atlanta, Georgia; Erie, Pennsylvani 

San Antonio, Texas; and Wise County, Virginia. 
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MICKOTEACHING AS A TEACHING METHODOLOGY 



Warren Kallenbach 

Associate Professor of Education 
San Jose State College 



The Development of the video tape recorder has added a major new dimension 
to the methodology of teaching* Now, for the first time, we can see immediately 
how we have performed in teaching. We cannot only see results at once but ran 
practice a given skill over and over in the microteaching studio until we are 
ready to try it in the classroom. We can view tapes or films of master 
teachers — or, at times, of teachers comparable and just a little better than 
ourselves and then practice this skill in both microteaching and actual class- 
room teaching sessions. (With training, v^e can use interaction analysis or 
the OScAR technique as we view. With apprppriate evaluative instruments and 
training, we can analyze and evaluate our own teaching performance or that 
of others.) 

The video tape recorder has been in commercial use only since 1956. Its 
portable versions came just a few years later. Prices for the portable 
versions have dropped from $20,000 plus to $14,000 to $10,000 to $3,500 and 
below. A home video tape recorder, not recommended for educational use, can 
be purchased for less than $2,000, complete with camera and playback equipment. 
Not only have prices dropped but quality and utility have increased. It is 
certain that scores even hundreds of portable video tape recorders will be 
purchased for school use in the next few years. How can they be put to 
their most effective uses in the schools? Or should the effectiveness 
criteria be put aside for a period of experimentation in each school so that 
perhaps totally unexpected instructional uses of the PVTR are discovered and 
tried? We could do both, of course, as some studies of classroom and in- 
structional use of video tape recorders have provided data that should not 
be ignored by new users. 

Perhaps I should describe what I mean by a video tape recorder. It is 
an electronic system whereby a television camera receives both visual and 
auditory images which are transmitted to the recording system, a system which 
retains images and sounds on tape much as an audio tape recorder retains 
sounds. Tapes vary from 1/2 inch to 2 inches in width and record from one 
half to two hours or more on each one. The video tapes can be erased as 
easily as audio tapes; an important safeguard to keep us from saving mediocre 
examples in our files forever. Electronic editing is possible with some 
systems which can mean a. great deal in the preparation of demonstration and 
ETV tapes. The tape recorder can play back on its own usually small, monitor, 
through any standard television set nearby or through television studio 
channels if the tape is compatible with the last or if it can be dubbed. 

It is possible to dub (i.e., tape from one tape to another) fYbm a one-inch 
to a two-inch tape if the systems are at all compatible. Regular 16mm films 
can be made frcai most video tapes; a considerable advantage for dissemination 
purposes. Color and studio quality video tapes are on the educational— use 
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horizons already. Portability of the tape recorder varies from difficult-to- 
move to backpack models which weigh 35 pounds and retail at $10,000. A model 
that can be moved easily by one man, through standard doorways, and up 
flights of stairs with no loss of recording or playback systems should be 

sou^t» 

The history of use of video tape recorders in instruction is a brief one. 
References in the educational literature ( Ed . Index) begin in 1959; appropri- 
ately enough an early reference appears in School Coach. Scattered research 

studies gradually appear. 

One major study of secondary intern teacher preparation at Stanford 
University led to experimentation with audio taping and filming of the class- 
room teaching of the interns for playback at seminars on the campus. . 

system was far from perfect in feedback (problems in lighting and pickup of 
pupils’ voices primarily) and the transition to video tape recorders— 
portable ones — was not long in coming. Having a large foundation grant and 
operating in the heart of the U.S. electronics industries, it was not long 
before several two-inch PVTR’s were obtained for use with the secondary 
intern program at Stanford. The Stanford project staff, headed by Drs. 

Dwight Allen and Robert Bush, developed the procedure of presentation of 
short, videotaped lessons or lesson segments (from 5 to 10 minutes) to small 
groups of from four to six pupils. These lessons were evaluated by both the 
supervisor and the pupils and were discussed by the intern candidate and the 
supervisor during the playback and critiquing period immediately following. 
Recommendations for one change of behavior was made for reteaching, which ^ 
again followed immediately with a comparable small group of pupils. Criti- 
quing again followed this videotaping, using the same procedures. The name 
micro- teaching was given to this process, a process that is being further 
refined and studied at Stanford University’s School of Education and else- 
where. ^ Micro-teaching offers the considerable advantages of immediate 
knowledge of results and opportunity to practice a given skill at once and 
until satisfactory performance is obtained and without disservice to 
classroom groups. 

Most of the research findings from experimental use of micro teaching 
have come from studies in Stanford’s Secondary Teacher Education Program 
and at the Stanford Center for Research and Development in Teaching. A tew 
other studies are now underway on other campuses. It is probable that scores 
of studies will emerge in the next year or two. 

A first major finding in the Stanford Program was that there were no 
significant differences in judged teacher competence between randomly- 
selected secondary intern teaching candidates (n=30) who had had summer 
student teaching and those who had participated in the micro-teaching program 
on campus. 2 Pre- and post-tests, on video tape, were judged, double-blind. 



1. Secondary Teacher Education Program. Micro-T eaching; A Dg ^ gr. ip tion . 
Stanford; School of Education, Stanford University, Summer 1966. 

2. Dwight W. Allen and James C. Fortune. An Analy_s_is. of Mic ro-Teaching; 
A New Procedure in Teacher Education . Stanford; School of Education, 
S*tanford University, c. 1964. 
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by hlghly~tralned evaluators. The tests consisted of short lessons presented 
by each Intern In the study. These findings from the summer program carried 
over Into the regular teaching year; success levels being judged with the 
same evaluation Instrument, the Stanford Teacher Competence Appraisal Guide. 
which had been demonstrated to be a highly reliable Instrument when used by 
trained observers. 3 

Validity of the Instrument was judged on the basis of the adequacy of 
Its sampling of the major teaching skill areas utilized by the Stanford STEP 
Staff. 

An Immediate consequence of the Stanford Intern Program findings was to 
drop the loglstlcally-dlfflcult summer student teaching and school aide 
programs In favor of an on-campus micro- teaching program for all such Interns. 
The design of this particular study Is being utilized In a two-year study at 
San Jose State College as applied to elementary Intern teachers. Findings 
In the first year of the San Jose study are similar to the Stanford findings, 
viz. , no significant differences between the control and experimental groups 
as judged from observations of their pre- and post-^test videotapes by highly- 
trained observers.^ A similar study is underway for 1966-1967 and includes 
a field follow-up using both STCAG and lOTA .^ - a more comprehensive teacher 
competence evaluation instrument. 

One advantage of microteaching is that it provides "An opportunity for 
those who are preparing to teach to obtain a liberal amount of practice 
immediately upon their entrance into training, under optimum conditions for 
the trainees and without endangering the learning of pupils."^ . 

It "Aims to break down teaching into simpler components so that the 
learning task will be more manageable for the beginner. 

The trainee focuses upon a certain aspect of teaching until he has 
developed a satisfactory minimum of skill before he proceeds to another 
skill.. In each session, he views his performance on video tape, receives 
a critique of it and of pupil evaluations, immediately reteaches the same 
lesson to. another small group of students with one recommended change and 
can repeat until successful. 

Several teaching skills have derived from the project and while these 
do not represent even a major part of a theory of teaching they at least 
bring some order to a rather nebulous area. 



3. Ibid . ’ 

4. Warren Kallenbach and Robert Tamonda. "The Effectiveness of Microteaching 
in the Preparation of Elementary Intern Teachers." Unpublished mimeographed 
paper, San Jose State College, 1966. 

5. Instrument for the Observation of Teaching Activities (IOTA) . Copyri^t 
1960, 196^ by Bradley, Kallenbach, Kinney, Owen, and V7ashlngton. 

6. Robert N. Bush and Dwight W. Alle.n* Micro-Teaching ! Controlled Practice 

in the Training of Teachers . Stanford: School of Education, Stanford 

University, 1964. 

7. Ibid. 
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Some of the. skills stressed for teaching in the Stanford Program are: 

1. Establishing Set - The establishment of cognitive rapport 

between pupils and teacher to obtain 
immediate involvement in the lesson. 

2. Establishing Appropriate Frames of Reference - Organizing and 

teaching from several appropriate points of 
view. 




3. Achieving Closure 



Attained when the major purposes, prin- 
ciples, and constructs of a lesson, or 
portion of a lesson, are judged to have 
been learned so that a pupil can relate 
new knowledge to past knowledge. 

Five or six other skills have also been isolated. 

Recently three important studies have been conducted using micro- 
teaching as a basis. 

One study^, tested the effects of self-feedback and reinforcement on 
the acquisition of a teaching skill, viz., the teacher’s use of positive 
reinforcement for pupil responses in classroom discussion. It was demon- 
strated that self-viewing (self feedback) by the intern was relatively in- 
effective (no significant changes in frequency of pupils’ responses during 
discussions) as measured in this study. The most effective training method 
existed when cue discrimination (pointing out of salient cues in teaching to 
which reinforcement should be attached) was added to the positive reinforce- 
ment given by the intern teacher’s supervisor during the viewing of the video 
tape playbacks. 

Another study^ tested the effects of feedback and practice conditions 
on the acquisition of a teaching strategy, viz., asking questions of pupils 
designed to elicit more information or more meaning for the pupil, this 
process being called probing . (Probing being broken down into clarification, 
critical awareness, redirection, prompting, and refocusing.) The effects of 
immediate feedback — massed practice; inraiediate feedback — distributed practice 
were tested. Outcomes favored mass practice — immediate feedback over dis- 
tributed practice — reinstated feedback. 



8. Frederick J. McDonald, ^ The Effects of Self-Feedback and Rein- 
forcement on the Acquisition of a Teaching Skill. Unpublished manuscript, 
Stanford University School of Education, Stanford, 1966. 

9. Dwight W. Allen, ^ Effects of Feedback and Practice Conditions 
on the Acquisition of a Teaching Strategy. Unpublished manuscript, Stanford 
University of Education, 1966. 
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The final study in this series^® tested the effects of modeling and feed- 
back variables on the acquisition by secondary intern candidates of a complex 
teaching skill - probing (used where questioning goes beyond first-answer 
pupil responses). 

In general, the experiment sought to determine which would be more 
efficients telling the person what to do (called symbolic modeling) or 
showing him what to do (called perceptual modeling) or some combination of 

these that Includes reinforcement and further discrimination training on 
the relevant cues. 

Six groups were randomly selected and different treatments were applied 
to each. 



Combinations of minimal and maximal symbolic and perceptual modeling 
were applied. Group six represented a combination of combined symbolic 
and maximal perceptual modeling and was determined to be the optimal 
training condition in achieving a favorable balance in use of probing 
techniques. It was clearly demonstrated that perceptual modeling excels 
over symbolic modeling. 



Even though self-viewing by intern teachers has not proved effective 
or as successful as some other modes in teaching, this may not be the case 
with experienced teachers. It may be quite worthwhile to test the use of 
the PVTR in the classrooms of experienced teachers in which they alone view 

the tapes and they alone decide what changes they wish to make in their 
teaching. 

This last approach was the basis of a study at San Jose State College: 
one called "Strategies in College Teaching", so as not to offend the sen- 

alarm the professors involved. Pre and post tapes of beginning 
psychology classes were made of five instructors all of whom who presented 
the same concepts for the taping. No significant differences were found 
between the tapes as judged by the same professors viewing each of the other 
four professor *s pre and post tapes on a better or less effective basis. 

This finding may be explained in that a total 50-minute class session was 
taped and, in some cases, the repeat taping brought out such attempts. to 
improve everything at once or such anxieties that quality actually went 
down upon the second performance. In the San Jose State College intern 
studies we fiiid that three or more microteaching sessions are necessary to 
overcome the first-time-on-television effect, — which we call the "cosmetic 
effect." Also in our microteaching projects we have learned to keep teaching 
sessions very short at first and to concentrate on one teaching skill at a. 
time, although this latter is very difficult for any supervisor of student 
or intern teachers at whom we formerly threw whole barrages of "goods" and 
"let’s imp rove’s." 



10. Michael E. J. Orme. The Effects of Modeling and Feedback Variables 
on the Acquisition of a Complex Teaching Strategy. Unpublished manuscript, 
Stanford University School of Education, 1966. 
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The uses of the PVTR seem limited only by our imaginations and finances. 
We could tape school board meetings for civics classes: tape kindergarten 

class performances to be played back to the PTA Meeting: tape You~Name-It 
for whatever. 

I prefer to concentrate on the improvement of teacher competence 
through microteaching even if I have to pras^tice via microteaching myself. 
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INDIVIDUALLY PKESCRIBED INSTRUCTION^ 
Richard C. Cox 

Learning Research and Development Center 
University of Pittsburgh 



Educators have long been aware of the need to develop a systematic 
approach to the educational process which is sensitive to individual 
differences among pupils. Plans and proposals that have been instituted 
for accomplishing this goal have been numerous during the past sixty years 
of American education. Many of these models xemain today only as note- 
worthly examples which either no longer exist in practxce or which have 
been so prostituted by the pressures of the present mass educational system 
that they are virtually unrecognizable. After-the-fact hypotheses about 
the apparent rejection, in practice, of these individualized systems in- 
clude criticisms concerning the lack of proper dissemination and evaluation 
activities, the lack of sufficient curricular materials and technology, and 
the lack of adequate support from various educational institutions. 

Educators, however, are not prone to abandon worthwhile goals. Evidence 
to substantiate this premise is apparent in the resurgence of discussion a- 
bout, and proposals for, the individualization of instruction. Part I of 
the Sixty-first Yearbook of the National Society for the Study of Education 
points out the importance and interest in the individualization of instruction. 
A recent conference at the Educational Testing Service was entitled "New 
Approaches to Individualizing Instruction." Much of this renewed interest 
seems to be a result of construction of Instructional materials, in measure- 
ment and evaluation procedures, in learning technology, and in technological 
devices which hold promise for the design of an individualized instructional 
system. 

The Individually Prescribed Instruction Project 

The Learning Research and Development Center at the University of 
Pittsburgh has identified as one of its major developmental activities a 
feasibility study of individualized instruction. In cooperation with the 
Baldwin Township School District in suburban Pittsburgh the Center has 
instituted the Individually Prescribed Instruction (IPI) system in the 
Oakleaf school for about one-half of each school day. At present, the 
Individualized program is used with the three content areas of mathematics, 
reading and science. School district procedures are followed in the in- 
struction of subjects taught during the remainder of the school day. 

The long-range goals of the IPI project art; 1) to restate certain 
elementary school curricula in terms of a cont'nuum of specific behavioral 
objectives which provides for the monitoring aru assessing of the prog’"‘ssive 



1. The research and development reported herein w’9 performed pursuant to a 
contract with the United State Office of Education, Department of Health, 
Education, and Welfare under the provision of the Cooperative Research Program. 
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development of each pupil* s competence in subject-matter areas; 2) to provide 
a variety of instructional materials and techniques to meet the individual 
needs of pupils; 3) to establish teacher functions and procedures to facili- 
tate individually prescribed instruction; 4) to develop a school structure 
and organization that permits the flexibility required for individualized 
learning; and 5) to provide operating procedures which are within the 
financial means of most innovative schools. 

As the IPI system was conceived certain assumptions were made to guide 
the staff in the development of the program. These include the following: 

1. One obvious way in which pupils differ is in the amount 
of time and practice that it takes to master given instruct- 
ional objectives. 

2. One important aspect of providing for individual diff- 
erences is to arrange conditions so that each student can work 
through the sequence of instructional units at his own pace and 
with the amount of practice that he needs. 

3. If a school has the proper types of study materials, 
elementary school pupils, working in a tutorial environment 
which emphasizes self-learning, can learn with a minimum amount 
of direct teacher instruction. 

4. In working through a sequence of instructional units, no 
pupil should be permitted to start work on ,a new unit until 
he has acquired a specified minimum degree of mastery of the 
material in the units identified as prerequisite to it. 

5. If pupils are to be permitted and encouraged to proceed at 
individual rates it is important for both the individual pupil 
and for the -teacher that the program provide for frequent 
evaluations of pupil progress which can provide a basis for the 
development of individual instructional prescriptions. 

6. Professionally trained teachers are employing themselves 
most productively when they are performing such tasks as in- 
structing individual pupils or small groups, diagnosing pupil 
needs, and planning instructional programs rather than carrying 
out such clerical duties as keeping records, scoring iiests, etc. 

The efficiency and economy of a school program can be increased 
by employing clerical help to relieve teachers of many non- 
teaching duties. 

7. Each pupil can assume more responsibility for planning and 
carrying out his own program of study than is permitted in most 
classrooms . 
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8. Learning can be enhanced, both for the tutor and for the 
one being tutored, if pupils are permitted to help one another 
in certain ways.^ 

The actual IPI program, is best described by a discussion of the essential 
elements of the system which include curriculum, materials and test develop- 
ment, and instructional procedures which include prescription writing and 
classroom management. 

Curriculum Development 

In developing the curricula in the IPI program, staff members, including 
teacher, psychologists, and subject-matter specialists, studied various cur- 
ricula presently available in an attempt to develop a sequence of learning 
experiences which would permit the flexibility necessary for individualizing 
instruction. The curricula needed to be expressed by carefully defined ob- 
jectives with each succeeding objective built upon the preceding where 
feasible. 

To facilitate the development of a sequence of objectives for grades 
1 through 6, each of the curricula was divided into levels, units and skills. 

A level Includes behavioral objectives from many separate categories re- 
presenting a level of achievement of a certain sequence of work. Each 
category within a level is called a unit, and within each unit the object- 
ives to be mastered are called skills. After general agreement between the 
Learning Research and Development Center staff and the classroom teachers 
concerning the content areas to be included, specific skill objectives were 
written for each of the three areas of mathematics, reading and science. 

Care was taken to Identify the units of work in each of the content areas 
and to write precise statements of the behavioral objective desired. The 
sequencing of objectives from the least to the most difficult was of primary 
concern to curriculum writers. Easier objectives were placed in units at 
lower levels; more difficult objectives were placed at higher levels. 

Material"^ 



Once the objectives for each subject area had been established, 
materiai|.s iaere developed or adapted to help carry out each objective. The 
teaching materials were primarily oriented toward self study since most of 
the pupil’s work is done independently. Lesson materials which best met 
these requirements were programmed textbooks and workbooks. No attempt was 
made to select any commercial series of textbooks or workbooks emphasizing 
a particular curriculum approach: rather, pages from textbooks and work- 
books were matched according to the behavioral objectives. 

In material selection and adaptation, the writers were careful to take 
advantage of skills taught at lower levels. Consideration was also given 
to the possible ranges of age, interest, and vocabulary of the pupils since 
pupils in the second and fifth grades could be working with the same objective. 



2. Llndvall, C. M. & Bolvin, J. 0., Individually prescribed instruction: 
The Oakleaf Project. Working Paper 8, University of Pittsburgh, Learning 
Research and Development Center, 196^. 
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Th© materials on which pupils work and from which teachers write pre- 
scriptions are arranged in two ways: by objective, and by individual page. 

The materials for each objective are placed in a folder; the accumulation 
of all folders making up a master file. From this file the teachers obtain 
iiiformation and sources for prescription writing. The loose-leaf pages for 
student use are stored on six library carts. In order to provide for cross- 
indexing, a list of pages with the source and behavioral objective of each 
page was made. This master file of the original materials which identifies 
the behavioral objective of each page assists the teacher in prescription 
writing. 

Diagnostic Instruments 

The evaluation of pupil progress is a crucial element of the IPI system. 
Information about pupil achievement provides the basis for continual planning 
of programs for pupils and for evaluating the effectiveness of instructional 
procedures. Four types of diagnostic instruments were developed to assess 
the strengths and weaknesses of each individual pupil on each behavioral 
objective. 

Placement Tests . Placement tests are administered at the beginning 
of each academic year to determine individual placement within each content 
area. These tests may extend over several levels of work; their purpose 
being to determine the level for each unit of work for which the pupil has 
demonstrated mastery or lack of mastery. A pupil is placed at the lowest 
level in which he has not demonstrated mastery. The placement test may 
not test all objectives in each unit but are intended only to yield a 
general profile of pupil achievement. 

B. Pretests . Because each unit and level of work includes several 
objectives, the pre-unit test was developed to identify which specific 
objectives within each unit a student knows or need to learn. The pre- 
test is designed to measure all of the specific objectives within a unit. 
These tests are administered before a prescription is written and indicate 
to the teacher what materials for which skills should be assigned to the 
pupil. If a pupil demonstrates mastery on each skill within the unit, he 
is not assigned any work in that unit. A pupil is assigned materials only 
for those skills for which he has indicated lack of mastery on the pretest. 
After a child has worked on these unmastered skills and his study indicates 
mastery, he is given the third type of diagnostic test, a curriculum em- 
bedded test. 



C. Curriculum Embedded Tests . The curriculum embedded test is a test 
of one specific objective within a unit of work. It measures attainment of 
that objective and serves as a short pretest of the next objective within 
the unit. The results of the curriculum embedded test aid the teacher in 
prescribing additional work on the objective or in assigning a new objective. 



D. Posttests . Mastery of all the skills within an assigned unit en- 
ables the pupil to take a posttest covering all of the skills within that 
unit. If a posttest, which is an alternate form of the pretest, indicates 
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lately, he is assigned the next unit. If lack of mastery is indicated on 
certain objectives, the student is assigned additional prescriptions of 

rsLoTpostteMf^*"^'*^^ allowed, to take 

Instructional Procedures 

Once the plac^nt testing is completed, the teacher determines where 
each pupil should begin instruction. On the basis of this diagnosis of a 
pupil s wetness, a prescription is developed for each child. *This pre- 

materials for the skill in which the child should begin 
??• prescription the teacher will generally co^siSr 

, the following factors: 1) the ability level of the child in each Lbject 

general maturity of the child, 3) the type of learner 
that the child seems to be, and 4) the pupil's general reaction to IPI. 

At the present time there are nine teachers and six clerks o- teacher 
aides assigned to the project. Seven teachers are assigned to homerooms 

subjectf? These se^r 

teasers also serve as a nucleus for the program in IPI. They are supple- 
mented by a math-science teacher who is responsible' for all of the IPI 
science as well as selected assignments in the mathematics program and a 
librarian who functions as a reading teacher or consultant forS’orthe 

daf »»cra of, students. Math and reading are scheduled one hour a 
and^h^^f^^ for each group. In order to share the additional teachers 

these subjects are never' scheduled in competition with 
each other, even for different groups. 

This is done"in^a”^a«e^ heginp work independently on the prescribed materials, 
a ^ provides space for all the students from 

a particular group, e.g. Primary I, Primary II or Intermediate. In this room 

tteir le!™?r®"® assigned to these students for most of 

their learning experience, augmented by the additional teacher depending upon 

^st o1 It and two, to four teacher assLt^ts! 

proceed through the prescribed materials with a 
^nimm of teacher direction and instruction. When assistance requiring <>x- 
tended explanations or instruction is required the team of teachers will 

instruction. The teachers res- 

ponsible for the instruction of a particular group also decide on the sharing 
of otuer respo^lblllties such as administration of pre and posttcsts large 

clMS period?"^””’ writing that may be necessary during ai?y 

In order to free the teacher for instructional decision-making tutoring 
and evaluation of student progress, the scoring of worksheets^ffests is 

tether <=>'il'lren themselves 'P. 



ine 



aspect of organization necessary for individualized 
instruv.tion is the provision for joint planning and evaluation for those 
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teachers assigned to a given group of students. During these planning sessions 
each pupil *s progress is analyzed and evaluated for each of the IPI subjects 
and suggestions are made as to type of prescription needs and for charting his 
program for the next week. It is also at this time that the assignments for 
teacher responsibilities are discussed and decided upon. A third factor 
decided or discussed during the planning session is the program to be followed 
during the "seminar’' period in math and reading. One of the scheduled periods 
each week in each of the areas is set aside for group interaction and parti- 
cipation. The planning for this activity is varied and complex and as an 
integral part of the overall program must be carefully planned and executed. 

Implications for the Public Schools 

Any implications cited here must be tempered by the realization that 
the Individually Prescribed Instruction project is basically a developmental 
study at the present time. Much of the data collected from the study has 
served the purpose of making decisions among alternative refinements of the 
individualization procedure rather than providing evidence concerning im- 
plications for future educational systems. However, there appear to be 
several factors in the IPI project which suggest some fundamental requirements 
for the individualization of instruction. 

School Organization 

Within any given grade level there is a wide range of pupil ability . 
Conventional grade level classifications do not facilitate varied rates of 
pupil progress. The idea of a first grade, a second grade etc. must be 
replaced with a system designed to permit each pupil to work at his own 
level of achievement. “Provision must also be made so that each pupil can 
proceed in each subject as soon as he achieves some given mastery criteria. 

A pupil, therefore, may spend anywhere from four months to several years 
acquiring mastery of skills typically identified as one year of schooling. 

School organization must also provide for a rapid and exact system to 
report information regarding each pupil’s progress so that, at any given 
time, the teacher can make a reasonable estimate about each pupil’s abilities 
and achievement in order to prescribe a teaching program congruent with his 
individual requirements. Schools should take advantage of the benefits of 
automated teaching de^rices and automated data collection techniques in order 
to achieve this goal.. 

Curriculum Development 

A curriculum for an individualized program must be based on a carefully 
sequenced listing of behaviorally stated instructional objectives. The 
achievement of each pupil should be defined by his position along this continuum. 

Curriculum materials must be geared to the instructional objectives and 
should allow pupils to proceed independently and with e minimum of teacher 
direction. The materials should also be such that the pupil will acquire 
competence in self-directed learning. 




Diagnostic Instruments should be an Integral part of the curriculum and 
must provide for the continuous evaluation of pupil progress. These Instru- 
ments must be content referenced in that they refer directly to the objectives 
stated In the curriculum and should provide Information concerning what the 
pupil does not know. 

Teacher Preparation 

Professional training must be provided so that teachers become Increasingly 
competent In the theory and practices of individualizing instruction. Teachers 
must be concerned with educational diagnosis and evaluation In order to guide 
the learning experiences of individuals , Teachers need to have a wide range 
of information available for assisting pupils working at varying levels. 
Subject-matter competence to teach one particular grade level will be in- 
sufficient preparation for the teacher in an individualized system. Emphasis 
in professional training must be given to the interpretation of test scores 
and other data so that the teacher can make appropriate diagnoses of pupil 
strengths and weaknesses. Each, pupil must be considered as an individual on 
a continuum of growth and it is the responsibility of the teacher to guide 
learning. 

In summary then, it seems possible to devise educational systems which 
are more sensitive to individual differences than past procedures have In- 
dicated. These systems will probably take a variety of form. However, there 
will be certain requirements for school organization, for curriculum develop- 
ment and for teacher preparation. Such Individualized procedures have many 
implications for educational systems of the future. 





REINFORCEMENT MENUS IN THE INSTRUCTION OF MENTALLY RETARDED CHILDREN 

Marvin F. Daley 

Director, Psychological Laboratory 
Utah State University 



jfy presentation has been called ’’Reinforcement Menus in the Instruction 
of Retarded Children.” In other settings the talk might have been titled, 
’’Human Motivation and the Environment,” or ’’Another Prosthetic Environment, 
or possibly "A Technique of Contingency Management.” 

Psychology Laboratories have long been developing techniques and 
principles applicable to the behavioral analysis of organisms throughout 
the Phylogenetic scale. The behavioral technology and concepts that have 
been worked out in animal laboratories are often applicable in the area of 
human behavior with little or no modification. 

These investigators required their human subjects to pull a lever, push 
a button, or engage in some other comparable motor activity to receive either 
an edible or a token. These direct applications have lead to such develop- 
ments as modifying infant verbal behavior (Rinegold, ^. , 1959) 
modifying cooperative behavior of young children (Azrin and Lindsley, 1956.) 
Such pioneer studies have promoted extensions of reinforcement principles 
developed in the animal laboratory to psychotherapy, human psychophysics, 
programmed instruction and virtually every other area of psychological 
research with humans. 

In recent years, our laboratories have been producing a rapidly growing 
body of research concerned with the motivational control of ben&vxoi.. 
concept of reinforcement has been developed by Premack, which states, ... 
for any pair of responses, the more probable one will reinforce the l®ss 
probable one (Premack, 1965).” A majvor supporting principle is the in- 
difference principle,” i.e., the reinforcement value of an event is inde- 
pendent of the factors producing the specific response probabilities. 

Taking Premack literally, any behavior can be used as a reinforcer of any, 
other lower probability behavior at the instant that the behavior is a 
higher probability one. In one of his early experiments (Premack, 1959) a 
Cebus monkey was placed in a chamber that had three response mechanisms 
attached to one wall. One or more of the mechanisms could be rendered 
inoperative. The monkey was given a period of time in the chamber during 
which response frequencies were determined with only one mechanism functioning 
at a time. It was found that the animal manipulated each response mechanism 
with different frequencies. The experimenter altered the environment so 
that the monkey had to operate the mechanism initially used leas^ frequently 
in order to have access to the mechanism initially used mo.s_t frequently. As 
a consequence of his contingency management, the response mechanism which 
the animal initially operated at a low frequency was now operated at a much 
higher frequency approximating that of the high frequency operandum. Since 
the original study, numerous others have been reported involving other 
organisms and responses such as turning wheels and pressing bars. 
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TI\6 significancs of fhoso findings when systematically applied to human 
behavior is overwhelming* First s it is not necessary when attempting to 
modify behavior to depend wholly on candy or trinkets as reinforcers as did 
the early investigators. It is necessary only that the Contingency Manager 
be able to identify what students are doing most frequently in a given 
environment. Homme, al,, (1963) observed children in a pre-school nupery. ‘ 
The children would run around the room, scream, push a chair across the floor 
and play games. All these behavi.ors were occurring most of the time. As one 
would expect, the instruction, ^come and sit down" would go relatively un- 
noticed. The investigators concluded that most of the high probability 
behaviors for these children were aversive to adults. Applying the Premack 
principle. Contingency Manager allowed the children to engage in "undesirable" 
behaviors contingent or dependent upon the subjects doing a very small amount 
of what the experimenters instructed them to do. A typical early contingency 
required the children to briefly sit quietly in chairs and look at the black- 
board. The behavior was followed quickly by the command, "everybody run and 
scream now." Immediate control over the situation was obtained. Time sitting 
in chairs was progressively. increased and training in school skills was begun. 
The experimenters reported that they were able to teach the entire first grade 
repertoire to these students in about one month of training. 

In another experiment, Homme (1966) was able to demonstrate that pre- 
school Indian children could make substantial improvements in learning English 
when taught by contingency management system. 

Still another twist for gaining better control over the contingency 
management has been to develop a kind of menu of activities that represent 
high probability responses (Addison and Homme, 1966). The menu has been 
put in a book form with systematic line drawings or stick figure sketches 
of the activities. The subject is presented with the menu before executing 
the task to be learned and is allowed to select from it an event which for 
him at that moment is a high probability event. Upon completion of the task, 
he is allowed to go to the reinforcement area and his selection is immediately 
made available. Homme has suggested that certain reinforcing events could 
be "daily specials" to be made available on a restricted basis. It should 
be noted that the implications and possibilities of such practical appli.cations 
are innumberable if emphasis is placed upon careful management of contingencies. 
With contingency management and the Premack principles, the teacher is in a 
position to deliberately, systematically, and efficiently modify behavior. 

At Utah State University we have initiated a series of experiments "« 7 hich 
will enable us to delineate areas of fruitful investigation directed at 
modifying behavior by careful contingency management. The experiments pre- 
sented here used mentally retarded individuals as subjects. 

Experiment 1 

One of our first experiments was designed to parallel the regular class- 
room situation. Five mentally retarded children from the Cache Valley Day 
Care and Training Center served as subjects. 
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Allen had a chronological age of 10 years, 1 month, a mental age of 5 
years 3 months, and Utah verbal language development age of 2 years 10 months. 
Case studies indicated he was easily distracted, tired easily, had a short 
attention span, and often displayed aggressive behavior toward other children. 

Loma, 8 years. 1 month old, had a mental age of 3 years 4 months and a 
Utah language age of 2 years 6 months. She tired easily and preferred 
passive activities. 

Patty had a chronological age of 9 years 0 months, a mental age of 4 
years 1 mx)nth, and a language age of 2 years 2 months. Her teachers reported 
that she was an aggressive, restless, child with a short attention span. 

Ruth was 10 years 1 month old, had a mental age of 4 years 0 months, and 
a language age of 2 years 3 months. She was considered moody and difficult 
to discipline. 

Tex is 11 years 2 months old, had a mental age of 4 years 0 months, 
and a language of 2 years 9 months. His teacher had referred to him as 
"...unreliable in any situation as well as a distraction. He is a burden 
to any group and his actions are dangerous to other children. 

The general concensus of opinion held was that these individuals were 
very difficult to motivate and they often emitted non-acceptable behavior 
in the school atmosphere. 

Armed as we were with our predecessors’ successes, the first step was 
made. Extensive observations were carried out producing 22 items which, 
because of their high frequency of occurrence, were considered to be high 
probability behaviors : 



Talking 


i . Dancing 


q. Using Colored Pencils 


Writing 


j . Walking 


r. Singing 


Coloring 


k. Drawing on Board 


s. Swinging on Door 


Drawing 


1. Telephoning 


t. Moving chair 


Reading 


m. Puzzle 


u. Erasing Blackboard 


Swinging feet 


n. Blocks 


V. Looking out Window 


Record 


0 . Jumping 


w. RE Book 


Hugging 


p . Drinking 





A menu depicting these activities in color was prepared by an artist 
and enclosed in a single book with one activity per page. This book in- 
dicating the reinforcing events is called an RE menu for short. 

The menu (Figure 1) was introduced to the class by encouraging them to 
name the activity involved while the RE menu was held before them and the 
pages turned. The children were then told, "You are going to be able to 
have times to do what ever you want to do that is in this book. Every time 
•you finish your work, you will get to look at our book and pick what you 
want to do for 4 minutes. I will tell you when your time is over to do 

what you want to do." 






The lesson activity was presented and upon completion of the exercise 
the children were told, "Here is the book. Now you can pick what you want 
to do for 4 minutes. Our things are here and only for this time when we use 
this book. No one else can use them." The children were shown the RE 
materials and after 4 minutes were instructed to replace the materials and 
were given the next exercise. From here on the children took turns being 
first to use the menu upon completion of the task. 

Lessons were taken directly from the Peabody language development kit 
(Dunn and Smith, 1965). The stated purpose of the kit is: a) "to stimulate 

the over-all oral language facility of the disadvantaged and retarded,*' b) 

"to develop their verbal intelligence through training," and c) "to enhance 
their school progress." Each lesson contained an average of 3 exercises, 
e.go, following directions, identification, classification, etc. In one 
case, a card is held up and the instruction given 'to the group to "pick out 
food eaten for breakfast," "pick out food eaten for snacks," etc. Experi- 
mental sessions were 60 minutes long and comparable to a regular class period. 

Initially an RE activity was made available after every 5 minutes o| 
work. By the 11th hour session, each task was now 30 minutes long. The 
experimenter-instructor was able to gradually shape larger and larger amounts 
of work to be accomplished before making available the RE menu. Thus, 
attention span and work output had been increased greatly. After 15 sessions, 
the children were re-evaluated with the Utah Verbal Language Development 
Scale. The range of improvement was 2 years 0 months to 2 years 6 months. 
(Figure 2). 

Another experiment was designed to explore the possibility of using a 
restricted menu as a device for making low probability task behavior into 
high probability reinforcing behavior. 

Girl subjects were selected from the Cache Valley Day Care and Training 
School. Each had a measure IQ of approximately 40 and a mental age of 6 
years on the Stanford-Binet. 

Two rooms x^ere used: one for performing the task and the other about 

25 yards away for housing the reinforcement area. 

Reading aloud two pages of textual material and writing two pages of 
programmed math problems were selected as the lox\r probability tasks. In 
order to have access to the RE area, a 90% correct criterion was employed 
for each task. After a mathematics assignment was completed, correct 
answers were indicated; and if less than 90% correct, the subject re-worked 
the entire set until the 90% criterion was met. In the case of reading 
aloud, immediately after an error the subject was corrected and then re- 
required to repeat the sentence. Upon successfully completing the task, the 
subject had 4 minutes access to the RE area. 

A restricted menu of high probability responses was employed for each 
subject in order that new menus might be more easily developed. Coloring, 
using cut-outs, playing with a Barbie doll and handling a cat were found to 
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be high probability behaviors for the representative subject reported here. 
(Figure 3). Light color sketches were made of these activities on 8 x 5 
cards and a menu prepared for each subject. Arithmetic and writing were 
selected as the low probability responses which could gradually come to 
replace the original items on the menu. Five separate menus were prepared 
for each subject (Figure 4). 

With each new menu (2 through 5) the components of the activities in- 
dicated on the card were gradually changed in the direction of mathematics 
for color and cut-out menu items and writing for doll playing items. The 
subject was not presented an earlier menu once it had been superseded with 
a new menu. The procedure of a unit of work, a unit of reinforcing activity 
was employed. At the completion of the experiment, 60 sessions, the subjects 
were doing units of work involving reading and mathematics in order to have 
the opportunity to do some reinforcing arithmetic. 

Rapid learning curves were produced by each subject. Their performance 
was worked by little variability on the response measures once asymptotic 
performance had been developed. Further, introduction of the new menus did 
not disrupt their behavior on any of the Response measures. 

Another striking feature of this data was the difference between the 
time it took to go to the RE area and the time it took to return to the task 
area. The latter was much faster. It was suggested that this result in- 
dicated the task itself may have acquired reinforcing properties (Figure 5). 

There were changes in behavior other than the measurable ones reported 
in the above experiments. Without exception every child began to demonstrate 
a higher frequency of socially desirable behavior; e.g.', temper tantrums no 
longer occu..red and teachers remarked on the improvement in social Interactions. 

One of the subjects was placed in junior high school and is still doing 
well in a special program. Further, none of the subjects have thus far shown 
a tendency for their enthusiasm to wane as the experiments progressed. 

Learning Centers of the Future 

It is not difficult to envision drastic changes in our educational system 
over the next decade. We have heard here at this conference a number of new, 
exciting, and provocative ideas about the classroom of toiaorrow, or indeed 
the lack of it as we know it today. 

One approach to modifying behavior that seems to be well suited to the 
motivational system which I have just presented is the computerized educational 
system. It seems to us that computers with their vast storage capacities 
while capable of handling numerous instruction consoles, at one time are 
uniquely compatible with a contingency management system. There is little 
question that any subject that can be taught by lecture can be more efficiently 
presented through a computer system. The day when carefully programmed 
computers enter into a dialogue with the student is very close. Computerized 
instruction which integrates programed instruction and contingency management 
into its general operation will be a most formidable system of behavior 
modification. 
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It appears that we have reached one solution to the problem of the ‘’un- 
motivated" individual — careful management of contingencies between low and 
high probability responses. In this sense the underlying theme from this 
paper has been behavioral engineering. 

It is in this context of behavioral engineering and education that we 
will go so far as to suggest that in most cases programmed environments are 
of necessity prosthetic environments. Further, it has been and still remains, 
our contention that modem science has finally acquired the ability to de- 
sign suitable environments for the behavior of a given individual as well 
as design the environments necessary for maximizing the individual's behavioral 
efficiency. 
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STRATEGIES lOR TEACHING STUDENTS TO ANALYZE PUBLIC CONTROVERSY* 

James P. Shaver 

Professor of Educational Research 
Utah State University 



Educational research is a young field, probably still in its adolescence 
if not in the later stages of infancy. School people are not yet able to 
find in research reports adequate answers to most of their pressing questions 
about educational practices. Questions about teaching students to^ think 
critically or reflectively, or put in the more specific terms of the title 
of this paper, teaching students to analyze public controversy, are no 
exception. There are few definite conclusions to be drawn from the availa e 
research. But we are not totally ignorant in this area. 

A few years *ago, I reviewed the research done on the teaching of 
critical thinking in the social studies. Viy reading of the research since 
the review was published in 1962 and has not turned up studies which would 
change ray conclusions at that time. 

Probably the most conclusive suggestion [for teachers] 
support^ d by the research reviewed here is that we should not 
expect that our students will learn to think critically as a 
by-product of the usual social .studies content. Instead, each 
teacher should determine what concepts are essential e.g., 
that of relevance— if his students are to perform the intel- 
lectual operations deemed necessary to critical thinking- 
such as, for example, the formulation and evaluation of 
hypotheses. Each of these should then be taught explicitly 
to the students. Utilizing what is known about transfer of 
learning, a further step can be suggested: Situations as 
similar as possible to those in which the students are to use 
their competencies should also be set up in the classroom, 
and students guided in application of the concepts in this 

context. 

It is when we begin to consider specific procedures and 
methods for carrying out thirj course of action that many 
questions arise. It has been the primary purpose of this 
review to emphasize that for the moment social studies teachers 
will not find the answers in reports of educational research. 

We can only hope that the future will be more fruitful. 

(Educational Research and Instruction for Critical Thinking. 

Social Education, 1962, 16.) 



*The work reported here was supported in part by the Cooperative Research 
Program of the U.S. Office of Education. 
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Process ^alysle^ Cambridge, Mess.: Addlson-Weslcy, 1951.) and expanded 
then to fit our theoretical descriptions .of the behavior that should occur 
when the two Instructional techniques were used. 

For each of our four experimental teachers, twelve discussions— six 
style— were selected for analysis fron a larger number recorded 
over a period of several weeks. Two-way analysis of variance was used 
to determine if different patterns of behavior did occur when the different 
techniques were to be In use. The differences were consistently large— so 
arge as to make tjid statistical tests of significance almost unnecessary. 
There also were significant differences among the teachers when they were 
to be ^ing the same technique. This apparently was a manifestation of 
the differing personality traits brought to the classroom by the teachers. 

In general, however, despite the differences within each technique, behavior 
similarities did not overlap between the two teaching styles. 



Having established the differences In teacher behavior for the two 
methods, we moved to the analysis of learning. First, the control students 
were compared with students In two control schools in similar communities. 

On tests of social studies knowledge and general reasoning ability, there 
were no significant differences; on tests developed to measure the use of 
concepts that the project Intended to teach, the experimental students 
scored significantly hl^er. (Some of these tests Involved scoring student 
participation In an interview or In a student discussion In order to have 
a testing context more ’’natural” than a paper— and-pencll test setting.) 

The next step was to compare the learning of students who were Involved 
In the discussion of public Issues through the socratlc technique with that 
of students whose exposure to the same Issues was through the recitation 
technique. No significant differences In learning emerged. This finding 
did not surprise us In the light of the numerous studies of teaching methods 
that 'have ended up with no significant differences or with contradictory 
results from one study to the next. There was, however, reason to suspect 
that proper analysis would reveal that students had reacted differently 
to the different teaching methods , even though when averaged together for 
the previous analysis, these differences in learning had been obscured. 

(See, e.g., W. J. McKeachie, Motivation, Teaching Methods, and College 
Learning. In Marshall Jones (editor), Nebraska Symposium on Motivation. 
Lincoln: University of Nebraska Press, 1961.) Consequently, we admlnls- 

tered a number of personality tests to our students In order to Investigate 
the interaction of student personality with teaching method. 

We found that none of our personality measures showjed consistently 
high correlations with our measures of learning. This, too, was In line 
with past research findings. Next, we classified each student according 
to the teaching method to which he had been exposed (socratlc or recitation) 
and according to whether he fell Into the bottom, middle, or top one-third 
of the distribution of scores on each of the personality tests administered. 
Having made these classifications, we used analysis of covariance (adjust- 
ing the posttest means for any differences on the pretest and on an IQ 

test) to test whether there was any interaction of method and personality 
In affecting posttest performance. 
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This summary of research provides the context for the curriculum 
development-research project out of which ny paper comes. The Harvard 
Social Studies Project, under the direction of Donald W. Oliver, was given 
responsibility for the social studies program of five groups of students 
(approximately 125 students) as they moved through the seventh and eighth 
grades of a suburban Boston school. Our Intent was to teach these students 
concepts that would help them to analyze public controversy more adequately. 
The instruction was carried out within the overall framework of a course 
in U.S. history which provided the usual content for the two-year social 
studies program in the experimental school. In fact, however, by the time 
we had administered our testing program and carried out our experimental 
teaching, only about one-third the usual amount of time was left over for 
the regular history coverage. (A complete description of the project and 
its findings is available in Donald W. Oliver and James P. Shaver, Teaching 
Public Issues in the High School . Boston: Houghton, Mifflin, 1966.) 

Our teaching centered on the analytic concepts that would help students 
to analyze public controversy, although we were also concerned with the 
constitutional principles and the governmental structure that provide the 
context for public debate. The analytic concepts were directed as three 
•types of problems people should confront in dealing with public Issues: 

1) the clarification of communication and language meanings, 2) determining 
matters of fact, and 3) political-ethical choices. These concepts were 
taught separately as part of an instructional unit, using civil rights 
cases (free speech, etc.) to Illustrate their appllca^on. Then the students 
were exposed to a number of substantive units (e.g., one on the problems of 
school desegregation) that presented public issues to which the analytic 
concepts could be applied. 

Students were assigned to two different instructional methods, labeled 
the socratic and the recitation , for the discussion of cases presenting 
public issues to be analyzed. With the socratic technique, the student 
was first encouraged to take a personal position on a policy to handle the 
issue presented in the case, and then forced to defend it. In particular, 
inconsistencies between his values and between his factual beliefs and his 
values were emphasized, and he was encouraged to take these into account 
in applying the analytic concepts and arriving at a defensible position. 

When using the recitation technique, the teacher began by reading the same 
case with his students. He then used a series of questions to get at matters 
such as possible policy positions, the data that might be relevant to making 
a decision, the language that might need to be clarified, and the value 
conflicts that might need to be confronted in arriving at a defensible 
position. The student was not, however, forced .to take a personal position 
and defend it. 

A set of observational categories was developed to determine whether 
or not the experimental teachers would be able to use both teaching tech- 
niques. The basic scoring unit was the simple sentence. These were scored 
in categories such as : Asks for Descriptive Statement, Suggests Evaluative 

Inconsistency, Gives an Evaluation, Disagrees with a Description, Shows 
Positive Affect, Shows Antagonism, and Tension Release. (See Table 1.) We 
began vjith a set of categories developed by Robert Bales ( Interaction. 
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Out of a total of ninety-one analyses i twelve were significant at the 
.05 level. At this level, only about five significant findings would be 
expected to enserge purely by chance. The results do not, therefore, seem 
to be a chance happening. At the same time, it is difficult to have confi- 
dence in any ont Interaction pattern until it has been reproduced In further 
research. 

What did the interactions look like? Figure One presents the inter- 
action effects of teaching method (or style, as we called it during the 
project) and Factor F (Friendliness) of the Gullford-Zimmerman Tenq^erament 
Survey in affecting performance on the Unit Test. The Unit Test score was 
a conqposlte score obtained by adding np the students* scores on a number 
of ”teacher-made'* tests utilizing a variety of typical classroom testing 
items and administered after each instructional unit. Note that students 
low on the "friendliness" factor (which is supposed to measure tolerance 
for hostile action and tendency not to dominate or be hostile) did better 
on the Unit Test when taught by the socratic method. And, while the recita- 
tion method produced better Unit Test scores as the "friendliness" scores 
increased, the opposite trend is observable for the socratic style with 
a slight rec very for the highest "friendliness" group. 

Speculating about the causes of such patterns of interaction is at 
best a tenuous process. But it could be that students low on Factor F 
concentrated their efforts on the more specific and manageable aspects 
measured by the Unit Tests in order to compensate for the frustration and 
the difficult to manage feelings generated by the adversarial stance of 
the. teacher using the socratic method. The performance of the "high 
friendliness" socratic group, while not up to that of the counterpart 
recitation group, might indicate that as tolerance for seemingly hostile 
action increases, the socratic becomes stimulating enough to provoke 
learning. The lower mean scote as conq>ared with the "high friendliness" 
recitation students may be accounted for by the fact that socratic teaching 
entails more argumentation and less attention to case details than does 
recitation teaching. The poor performance of "low friendliness" students 
with the recitation method is hard to account for, unless it indicates that 
the student *s hostility stood in the way of the classroom participation 
that can be helpful in learning specific outcomes. 

Examination of this one interaction must suffice to indicate the 
general nature of the interactions we found. The patterns of interaction 
varied with the tests involved, and each pattern was difficult to interpret. 
Nevertheless, the results indicate why many researchers are coming to see 
this type of investigation as among the most provocative and promising in 
research on teaching. 

Despite the lack of clear and accumulated research findings that would 
provide a basis for educational decisions, the evidence of teaching method- 
student personality interactions has tremendous implications for teaching. 

It seems clear that the effects of different teaching methods (including 
programed instruction, for which there is a pittance of studies investi- 
gating the effects of student personality on learning) will vary depending 
upon the traits of the student being taught. In the past, educators have 
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been cognizant that differing levels of student intellectual ability might 
be an important variable for understanding the effectiveness of differing 
teaching methods (although ’’ability grouping” has not produced clearcut 
results). Perhaps it is time for teachers to seriously consider grouping 
students by personality types and adapting teaching methods to the different 
groupings. (For an interesting report of such an attempt on the college 
level, see George A. Stern, Environments for Learning. In Nevitt Sanford 
(editor). The American College » New York: Wiley & Sons, 1962.) Research 

cannot yet say what methods will be appropriate for what students. But 
if educational practice always waited on definite educational research, 

change would rarely occur. 

If the schof J.’s aims include teaching for the analysis of public 
controversy, the necessary strategy for optimum learning seems clear. The 
teacher must identify those concepts that the students are to learn and 
teach these explicitly. (This does not mean that ’’inductive methods can 
not be used. It does mean that the ’’induction” of the concepts will not 
be left to chance or expected from the usual classroom materials.) The 
students should also be guided in the application of the concepts in 
situations as similar as possible to those in which it is hoped the con- 
cepts will be used, and helped to develop general rules for fheir use. 
However, when it comes to the teaching methods that will be most effective 
for teaching the student to apply the concepts to public issues, research 
provides few clearcut answers. One tning seems evident, however. We 
cannot expect our teaching methods to have the same impact on all students. 
Allowance must be made for temperament as well as for intellect. 
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Table 1: Brief Definitions of the Categories in the Observational 
System for the Description of Teacher Style 



Affective, or Socio-emotional . Categories 

SOLIDARlTY’^’^Status raising language or tone of voice; strong approval or 
acceptance of another person. Often indicated by enthusiastic acceptance 
of another's ideas. 

LOW POSITIVE AFFECT — Signs of mild approval or acceptance of another person, 
or of his ideas. 

TENSION BELEASE-^-Action interpreted as tension reducing or attempting tension 
reduction, e.g. , laughing or telling a joke. 

TENSION— Behavior indicative of a state pf tension, such as stuttering or 
becoming tongue-tied. 

IX)W NH >ATIVE AFFECT— Statements or acts indicating mild disapproval or re- 
jection of miother person, e.g., disbelief or, skepticism about, a statement 
by the other person. 

ANTAGONISM — Deflating, derogatory, or highly negative statements or actions.. 

NEUTRAL— Acts or statements with no affective message discrlminable by the 
observer. 

Cognitive Categories 

SUGGESTS INCONSISTENCY— An attempt to lead another person to see inconsistencies 
in his values, claims, or definitions. 

EVALUATIVE — Statements which evaluate events, l.e., statements of like or 
dislike, rl^t or wrong, good or bad. 

DESCRIPTIVE— Statements which describe events, l.e., make claims about what 
reality is like, was like, or will be like. 

REPEATS, SUMMARIZES, FOCUSES— Statements that restate what has happened during 
the discussion, or bring attention to what is happening or going to happen. 

CLARIFICATION — Statements that attempt to clarify the content of the dis- 
cussion, l.e.,, clear up the meanings of statements or specific words. 

ANALOGY— Statements setting up for consideration a situation similar to the 
on4i under discussion. The situation set up may be hypothetical or one which 
it is claimed existed, exists, or will exist. 

NON-COGNITI^. E— Acts with no cognitive— Including procedural— message dis- 
crlminable by the observer. 
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Table 1 Continu-id 



'*^rocedural Categories 



DIKECIS task oriented BEHAVIOR-Statements 

which is in line with the task of the group, or at delineating wha. th 
behavior will be. 



CONTROLS DEVIANT BEHAVIOR-Statements directed at controlling behavior whidi 
detracts from the accomplishment of the group task. 
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STUDYING TEACHER BEHAVIOR WITH THE OScAR TECHNIQUE 

Donald M. Medley 
Educational Testing Service 



nf ® growing interest on the part \ 

tlnouichoH workers in the study of the teaching process as dis- ’ 

tingu shed from the learning process. Research in teaching has one important 

i‘'^is frdi f ft learning; you can't do it with rats. In f^t, 

tLt oractJcanv^*^!! teaching anywhere but in the classroom, 

happen to ourif T®"® anything the researched 

happen to find out is much more likely to be useful than something they 

discover *out rats in a maze— or at least its usefulness is more obviLs. 

teaching has another valuable by-product; many of the Instruments 

used by researchers to study teacher behaviddan alto beds^ by supe^Sod 

behaviod Interested in changing than in studying teLher 

I want to discuss the work of our own group. Since we have been 
concerned with instrumentation, instrumentation will be the 

1 will pay particular attention to the 
i’' ‘‘'® teaching may prove useful in 

dto:tld3rdo“dT IPlarventlon in the 

tha Ikat used by most other groups interested in 

n..v nai h process in that it has been basically a measurement approach. 

Our goal has been to develop procedures for obtaining objective quantitative 

orth3'’btols If df 7‘ 1“ of a "inlmum number of dimensions, 

observation. Behavior profiles so obtained could be 
used to study differences in behaviors of effective and Ineffective teachers 
c anges in teachers during training, to measure the effects of 
training on teacher behavior, and for a host of other purposes. 

”*'1®'' 1® acronym for "Observation Schedule and Record," 

beha3ln3« ®^®f contains a list or taxonomy of 

nmvTdo observer is to look for during a classroom visit, and also 

3ffn3e < recording the frequency of occurrence of each item. An 

“®"® 1° '>® observed in such a way that the 
CU..S on which they are discriminated will be as simple and as eLllv re- 
cognized as possible, so that the observer will need a minimum ^unt of 

sophistication and special training. His task is to record what he sees with- 
out judging or evaluating. c wxtn 

htict ^ separate step from observing, and is essentially a clerical 

task of combining items into sets or "keys" each of which describes an in- 

behavior. Factor analytical methods are then used to 
derive dimensional scores from the keys. 
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Our first stu^ using the technique was made in New York 
schools with first-year teachers who had been trained in the municipal coll g 
system. In addition to the observational data we also obtained each teacher s 
scores on several criteria of effectiveness. 

To illustrate one of the uses of objective behavior measupments, we may 
examine the data of this study to see what we can 

meaning of various measures of teacher effectiveness often used as c^teria 
In arch stucies, as well as for selection, promotion, merit pay, and the 

like. 

! 

Ratings of teacher coiirpetence by principals' and pupils' reactions to 
the teacher were found to favor teachers whose classrooms were 
orderly. Pupils also tend to prefer teachers who eiq>ha8ize veroal actmtles 
rather than non-verbal ones, but principals' ratings are unrelated , 

dimension. Neither the pupils' nor the principals opinions ® 

competence had any detectable relationship to how much the pupils learned 

from the teacher. 

Since much research in teacher effectiveness has used ratings of com- 
petence as a criterion, these findings have definite bearing on the inter- 
pretation of the results of such studies. And slnta ratings ®te a ®° ^ 

used in promoting teacher determining who is to ge'. merit P®y» ® ^ 

suggest that we may be rewarding teachers more for keeping pupils quiet and 

happy thau for helping them learn. 

Teacher self ratings, on the other hand, did indicate how effective 
the teachers were in helping pupils learn, but not how we e 
or principals like the teacher. Teachers who rated the^elves as 
effective tended to permit less small group work than those who rated them- 
selves as less effective. However, when mean pupil gain was 
teacher behavior it did. not show any relationship either to social - 

tion or to verbal emphasis; and its relationship to emotional climate, though 
positive, was too small to justify any confidence. 

The second study t want to mention involved student teachers in a c^us 
school* 'two observations were made of each student teacher at the beginning 
an^t tS^ end of the semester with the principal purpose of studying changes 
S theif^havior. So far as we were able to ascertain, no 
ever been done before— no attempt to make objective caparisons beWeen th 
behavior of the same teacher at different periods in his career had ever been 

made. 

l.e OSCAR used in this study was a rather elaborate, one; it was 8“'®^ 
nn 35 kevs and vielded 8-dlmensional behavior profiles. To get so^ 1 
the of S;l.ges which took place, we noted that 12X 

in behavior observed represented changes, and that 46% represented stab 
differences between different teachers— a ratio of about one to four. 

Of the observed changes, about half were ’ 

presented instances in which students teaching under virtually identical 



48 



conditions— same supervisor, cooperating teacher, ^d 

ODOosite directions. About a fourth represented changes uniform for .all 
student teachers; these may be interpreted as representing the effects 

the experience itself. 

What was the nature of these changes? 
climate were the most stable of all. Teachers use of approval and dis 
approval showed almost no change, nor was there any appreciable gain 
teadier’s awareness of pupils* feelings. 

Ihe pattern of pupil behavior changed significantly; pupil 
creased but the amount of initiative or freedom the teacher ^ 

not Ihe teacher's own role also changed significantly. He increased i 
noc. ine LCd^u r: Unth nf his own and his pupils* behavior), 

presence (appearance of control both ot nis o^^ informative 

His teaching style became less imaginative and original, but more 
— dealt with more content. 

I think these two examples illustrate that information about areas 
where both our ignorance and our need for knowledge are great can be °b- 
fainedTstudlef using the OScAR technique. Both of these studies used 
OSCARS which were hastily constructed for those studies alone. 

Currently we are at work on a carefully constructed instrument »bich 
may prove useful in a number of projects-some of which may be done by others. 

OScAR 4V is the first part of this omnibus instrument, a part dealing 
only wi^teacher^ verbal behaviors. It provides for the coding of each 
teacher utterance into one of 50 different categories. The cues 
the coding are extremely simple; classifying any verbal behavior 
fserierof no more than six elementary yes-no decisions. An example of 
a teacher statement needing the maximum number of discriminations is the 

following: 

"A funny thing happened to me on the way to school." The discriminations 
needed to code this are as follows: 



1 . 

2 . 

3. 

4 . 

5. 

6 . 



Did teacher say it? (Yes) 

Did it solicit a pupil response? (No) 

Did it deal with feelings, values, desires. ^No) 

Did it deal with content the pupils are supposed to learn. (No) 

Did it constrain or limit pupil behaviors? (No) ^«^oTnon^'? 

Does it belong in the same category as the last teacher statement. 

(ho) 



The types of behaviors discriminated in the system look general like 
thinnfLe shLld know about a teacher. Statements are classified according 
to wLther they relate to motivation, management, or subject matter; questions 

fre cl^“fi ^according to whether they call “S::s’a 

^>le» mini 1 a chance to originate his own answer, or ask him t . 

previous pupil comment. Information is obtained about how " 

of a lessL is, how intense the hostility or ® 

the extent to which the teacher dominates the pupils, what kind of pupil 

beL^or he encourages, how he uses criticism and objectives feedback, and so on 



If we had used this OScAR in the two studies I have described, it seenjs 
likely that we would have learned much more both about criteria of effect- 
iveness and about how teachers* behavior changes with experience. One of 
my main reasons for being here is to encourage others to use OScAR in studies 
of these and other problems. 

Jfy other purpose is to suggest that this kind of instrument might also 
be very useful in projects whose purpose is not do research but to implement 
educational change. If the project is dependent upon producing changes in 
teacher behavior— whether by better training .or supervision, by introducing 
new curricula or materials, or by reorganizing the administrative structure 
to free teachers to teach better, systematic measurements of teacher behavior 
can make an Important contribution to its success. Such measurements can 
tell you whether or not teachers do in fact teach differently after the 
change; they may tell you why some teachers improve and others get worse after 
the innovation; if they are used for feedback they can be Instrumental in 
expediting the changes teachers need to make in their behavior if the new 
program is to achieve its goal. 



BEYOND PROGRAMED INSTRUCTION.... 
Lincoln F. Hanson 



Professor of Education 
Rockland Community College 
Suffem, New York 



For ten years now the field of programed instruction has been plagued 
with the bandying about of borrowed terras that may or may not turn out to 
have proven utility in the advancement of knowledge about learning. 

In common with most teachers we have neglected one part of the process 
in a fashion almost as incredible as it is dangerous. This neglected bu 
crucial aspect of learning is, very simply, the process of listening to the 
student. Most of us acknowledge the importance of student responses in 
developmental testing, f not new (but regularly not employed) technique 
available to the publisher and useful to the teacher. Only a small handful 
of programs we encounter shows any serious evidence of developmental tes ng 
as it could and should be practiced. A small amount of additional develop- 
mental data may turn up in the few teachers' manuals worthy or the name. 

And a few more publishers are known to have data which, for one reason or 
another, has not reached the li^t of day. 

What we have in mind, however, goes well beyond all extant techniques 
and, we propose, stands to benefit the student, the teacher, the administrator 
the publisher, and especially the author. Furthermore, this modest technique 
will (not should, not may) provide a greater and more continous source of 
error signals than any educational feedback technique now known to exist. 

The technique is herewith dubbed "the carbon paper ear" and its ad- 
vantages will either be self-evident or it will properly die. 

One of the few advantages of programed over conventional educational 
materials is the opportunity for the student to demonstrate an interaction 
with the printed word. Yet, of what does this activity usually consist? 
Saccadic eye movements and sub-vocal muscular activity in the tongue and 
throat have all been reported— and can be operationally demonstrated if they 
do occur. A writj:en response may be made. But what is really going on. 

We usually wait for something, we call a test to reveal the nature and 
extent of the brain activity which lies beyond ready measure. What a 
travesty on the human learning process! So we. invent the open-ended and 
essay tLts, which, with considerably less stimulation on our part, elicit . 
a greater proportion of student activity. 

And now we lose objectivity and enter the mystical realms of sub- 
iective. Not only because the student writes what he is thinking (^ 
forbid!) but even more dangerously, because. we must now 
reconciliation of student activity with teacher intent. All too often 












graduate assistant injects yet another ''mind” into the process! (All conscien- 
tious professors and teachers in less economically favored school systems 
automatically exempted.) 

If our goal was student activity~of the highly individualized sort that 
students de'monstrate so weU in non-class aspects of their behavior— why do 
we delay this crude measuring process by even a day, say nothing of the weekly 
quiz, the monthly test, the mid and semester or even final exam? 

"But how can the student respond immediately and individually if he doesn't 
even know what I want?” 

Try him! — and listen. 

"But I have 30 students! I can't listen to them all.” 

All right— listen on paper—! 

"What? How?” 

Let him react to every point you wish to make. Assume you have a programed 

text. Do you presently let the student fill in the blanks— or otherwise write 
in the text? 

"No— we cannot— the books are too expensive and must be used by other 
students.” 

Fair enough. Do you have inexpensive lined paper? Yes — and can you pro- 
vide a modest supply of carbon paper? (We trust) 



Each sheet (conq>lete with carbon paper under) is divided thus: 



NO. 


RESPONSE COMMENT SlTRfn?STTnNR 
























► 






' 


’ * * 



In the first column go numbers of enough response items to enable you and 
the student to keep track. Three or four key items nuiobered per page appears 
adequate. 



In the second column the student responds to the program. If he covers 
the answers and makes any errors, he draws a single line through his Initial 
response and adds the correct response— "copied”. If you will, from the answer 
section of the frame. 



o 



52 



So far— a fairly standard procedure, .but... 

In the third column the student writes any comment he may have, we 
would ^gest that raw emotion be encouraged here Insofar as you may be able 
to persuade the student to express it in writing. 

A modest number of teachers and a larget'^number of program 

alrea^eSloyed this additional — • program, 

capturing what is now proposed as mandatory for every user or ev ry i 

Tn rhe last column we simply shift the burden of communication (further 
towarfa shaded position only.) by asking for "const^ctive suggestions which 
might better draw out the response you think the author want. . 

"Hhat do I do with all this student-produced written matter." 

1. The students keep one copy. (Just a simple record of performance!) 

2 ■'n a conventional lock-step class the teacher collects the carbon 

c;pii of tfeda^s "assigned" frames and by simply fanning them on the 

desrseL a ratLr unusual profile which indicates- 

a. whether anyone had any reactions 

b. who had what reactions 
who had problems and 



c. 

d. 



:ro rdeTgrerti-. many of which will be related to (c) 



3 If the nature of program use has achieved its logical goal of 
individual student progress rate, the carbons are dropped into each _ 
s?fdlit“rfUrto be htld against the day of ift^t 

file folders may never need to be touched— any My ® P® 

L hurdles such as conventional tests are f teacher ^ 

such cumulative diagnostic and resource ^ 

presents a novelty deemed well worth the exi...3 fil 

Stated another way. in contrast with most homework assi^ments or dally 
-exactlv where the misunderstandings were. 

::hfr::uyVas“Mbi::r^^^^^ 

the time of the whole class with a unilateral engagement oi tne 
time and energy. 

we can only conjecture as to f * 

met their (however iLdequately) before he 

"heard" from every student, on m p confidence to corn- 

started speaking. Maybe now a eac foster intelligent discussions with 

pletely new areas! "\“er^^ dfscissioM. student A's 

SstionlfcSSd ffr af a possible answer to student X’s problem or comment 
on item //531. 
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But this sketch is only a suggestive start. 

4. A department or sectional head may now see specific reactions of 
students to the text, positive quite as readily as problem. 

5. Periodically the teacher might bundle up his carbons and ship them 
off to the publisher— who, for perhaps the first time In history, would 
have a flow of pertinent Information, unlike any normally available, on 
which to base his decisions to revise and reprint. 

6. Last In the chain, the author now may have not only the reactions of 

a polite class of his own who have been "exposed to” his text In developing 
form, but a continuous monitor of student by student constantly (and 
probably repetltously) reacting to each point In the text. While such a 
bsrrage mi.ght become formadlble, there Is a possible advantage as and If 
his audience widens beyond or changes from the original target population. 
Further, he Is also getting a continuous flow of suggestions, of which 
even a small fraction could shape both book and author perceptibly! 

Some of our visitors have asked If the carbon paper ear Is copyrighted or 
patented. Nonsense! Good teacners have been sensitive listeners for years. 

A few program. writers and editors have similarly found how much they could 
learn from their developmental test students, even as you and I recall how much 
we learned from our first days at the front of a classroom. 

No, one reason for taking the liberty of projecting a local "technique” 

Into this space Is to ensure the widest possible dissemination of an obviously 
promising and almost Incredibly simple method of "listening" to our learners. 

A couple of perspicacious teachers have already noted that this technique, 
definitely derived from programed Instruction, and superficially advantageous 
to "linear” materials. Is, In fact, amenable to any text material whatsoever. 

It has been tried with math exercises. 

It has been tried with grnnmar exercises. 

It has even been tried successfully with paragraphs of extant conventional 
texts. 

And at any moment we expect to hear that someone has tried It with a 
novel or a poem — . 

Why not— where can we no listen with profit? 

The day we stop learning is the day we should stop teaching. 
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Figure 1 

SAMPLE STUDENT RESPONSE TAKEN FROM A GENERAL PSYCHOLOGY COUI^E 
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